Improvements in speech understanding accuracy through the integration of hierarchical linguistic, prosodic, and phonological constraints in the jupiter domain

This paper explores some issues in designing conversational systems with integrated higher level constraints. We experiment with a configuration that combines a context-dependent acoustic front-end, using MIT’s SUMMIT recognizer, with ANGIE, a hierarchical framework that models word substructure and phonological processes, and with TINA, a trainable probabilistic natural language (NL) model. Working in the Jupiter weather domain, we develop a computationally tractable system which incorporates higher level linguistic, prosodic and phonological constraints together in the second of a two-pass strategy. Experiments are evaluated using a new understanding performance metric, and the new integrated system achieves up to 17.1% relative reduction in understanding error and 15.4% reduction in word error. In addition, we investigate the possibilities of a twopass system which relies on the first stage for pruning based on syllable-level constraint, and applies linguistic and prosodic knowledge largely at the second stage.